The seed for this crawl was a list of every host in the Wayback Machine This crawl was run at a level 1 (URLs including their embeds, plus the URLs of all outbound links including their embeds) The WARC files associated with this crawl are not currently available to the general public.
The seeds for this crawl came from: 251 million Domains that had at least one link from a different domain in the Wayback Machine, across all time ~ 300 million Domains that we had in the Wayback, across all time 55,945,067 Domains from https://archive.org/details/wide00016 This crawl was run with a Heritrix setting of "maxHops=0" (URLs including their embeds) The WARC files associated with this crawl are not currently available to the general public.
The seed for this crawl was a list of every host in the Wayback Machine This crawl was run at a level 1 (URLs including their embeds, plus the URLs of all outbound links including their embeds) The WARC files associated with this crawl are not currently available to the general public.
The seed for this crawl was a list of every host in the Wayback Machine This crawl was run at a level 1 (URLs including their embeds, plus the URLs of all outbound links including their embeds) The WARC files associated with this crawl are not currently available to the general public.
This "Survey" crawl was started on Feb. 24, 2018. This crawl was run with a Heritrix setting of "maxHops=0" (URLs including their embeds) Survey 7 is based on a seed list of 339,249,218 URLs which is all the URLs in the Wayback Machine that we saw a 200 response code from in 2017 based on a query we ran on Feb. 1st, 2018. The WARC files associated with this crawl are not currently available to the general public.
The seed for this crawl was a list of every host in the Wayback Machine This crawl was run at a level 1 (URLs including their embeds, plus the URLs of all outbound links including their embeds) The WARC files associated with this crawl are not currently available to the general public.
The seed for this crawl was a list of every host in the Wayback Machine This crawl was run at a level 1 (URLs including their embeds, plus the URLs of all outbound links including their embeds) The WARC files associated with this crawl are not currently available to the general public.
Survey crawl of .com domains started January 2011.
Topic: webcrawl
The seed for this crawl was a list of every host in the Wayback Machine This crawl was run at a level 1 (URLs including their embeds, plus the URLs of all outbound links including their embeds) The WARC files associated with this crawl are not currently available to the general public.
Survey crawl of .net domains started December 2010.
Topic: webcrawl
COM survey crawl data collected by Internet Archive in 2009-2010. This data is currently not publicly accessible.
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl413.us.archive.org:survey from Sat Dec 21 07:47:29 PST 2013 to Sat Dec 21 00:52:11 PST 2013.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl800.us.archive.org:survey from Fri Jan 29 19:24:19 PST 2016 to Fri Jan 29 15:37:03 PST 2016.
Topic: crawldata
Survey crawl of .net domains started October 2011.
Topics: webwidecrawl, net
Survey of .org domains. This data is currently not publicly accessible.
1.6M
1.6M
Feb 22, 2019
02/19
by
Internet Archive
web
eye 1.6M
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl836.us.archive.org:survey from Fri Feb 22 02:17:40 PST 2019 to Thu Feb 21 23:46:50 PST 2019.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl344.us.archive.org:survey from Sat Sep 30 10:02:40 PDT 2017 to Sat Sep 30 03:34:44 PDT 2017.
Topic: crawldata
939,890
940K
Jan 13, 2019
01/19
by
Internet Archive
web
eye 939,890
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl339.us.archive.org:survey from Sat Jan 12 18:55:42 PST 2019 to Sat Jan 12 19:13:13 PST 2019.
Topic: crawldata
413,977
414K
Dec 10, 2018
12/18
by
Internet Archive
web
eye 413,977
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl836.us.archive.org:survey from Mon Dec 10 09:04:48 PST 2018 to Mon Dec 10 06:28:38 PST 2018.
Topic: crawldata
1.5M
1.5M
Jan 18, 2019
01/19
by
Internet Archive
web
eye 1.5M
favorite 1
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl838.us.archive.org:survey from Fri Jan 18 06:42:06 PST 2019 to Fri Jan 18 00:46:03 PST 2019.
Topic: crawldata
714,440
714K
Jan 26, 2019
01/19
by
Internet Archive
web
eye 714,440
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl824.us.archive.org:survey from Sat Jan 26 15:02:34 PST 2019 to Sat Jan 26 08:47:21 PST 2019.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl838.us.archive.org:survey from Fri Jan 5 09:25:00 PST 2018 to Fri Jan 5 01:50:03 PST 2018.
Topic: crawldata
1.3M
1.3M
Jan 29, 2019
01/19
by
Internet Archive
web
eye 1.3M
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl818.us.archive.org:survey from Tue Jan 29 11:32:54 PST 2019 to Tue Jan 29 05:46:54 PST 2019.
Topic: crawldata
713,079
713K
Jan 24, 2019
01/19
by
Internet Archive
web
eye 713,079
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl818.us.archive.org:survey from Thu Jan 24 00:44:19 PST 2019 to Wed Jan 23 18:36:19 PST 2019.
Topic: crawldata
654,500
655K
Nov 30, 2018
11/18
by
Internet Archive
web
eye 654,500
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl344.us.archive.org:survey from Thu Nov 29 20:01:41 PST 2018 to Thu Nov 29 17:30:37 PST 2018.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl339.us.archive.org:survey from Sat Oct 14 11:17:53 PDT 2017 to Sat Oct 14 04:45:39 PDT 2017.
Topic: crawldata
696,971
697K
Jan 26, 2019
01/19
by
Internet Archive
web
eye 696,971
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl824.us.archive.org:survey from Sat Jan 26 10:01:13 PST 2019 to Sat Jan 26 05:36:23 PST 2019.
Topic: crawldata
633,107
633K
Nov 26, 2018
11/18
by
Internet Archive
web
eye 633,107
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl825.us.archive.org:survey from Mon Nov 26 06:50:00 PST 2018 to Mon Nov 26 06:16:37 PST 2018.
Topic: crawldata
364,567
365K
May 25, 2020
05/20
by
Internet Archive
web
eye 364,567
favorite 0
comment 0
"Internet Archive crawldata from feed-driven by 1.2 million top ranked domains from data.domainrank.io - captured by crawl421.us.archive.org:survey_00010 from Sun May 24 19:26:13 PDT 2020 to Sun May 24 15:10:12 PDT 2020."
Topics: survey_00010, crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl836.us.archive.org:survey from Wed Oct 26 19:56:48 PDT 2016 to Wed Oct 26 15:40:51 PDT 2016.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl344.us.archive.org:survey from Mon Oct 16 01:15:40 PDT 2017 to Sun Oct 15 18:45:40 PDT 2017.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl824.us.archive.org:survey from Sat Oct 14 11:26:47 PDT 2017 to Sat Oct 14 04:54:28 PDT 2017.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl805.us.archive.org:survey from Fri Feb 19 11:46:14 PST 2016 to Fri Feb 19 12:08:16 PST 2016.
Topic: crawldata
3.5M
3.5M
May 15, 2018
05/18
by
Internet Archive
web
eye 3.5M
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl841.us.archive.org:survey from Mon May 14 17:51:22 PDT 2018 to Mon May 14 16:01:52 PDT 2018.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl808.us.archive.org:survey from Sat Dec 27 23:31:46 PST 2014 to Sat Dec 27 21:07:24 PST 2014.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl413.us.archive.org:survey from Tue Jun 3 04:58:57 PDT 2014 to Tue Jun 3 01:09:27 PDT 2014.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl344.us.archive.org:survey from Wed Oct 11 23:22:24 PDT 2017 to Wed Oct 11 16:29:56 PDT 2017.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl453.us.archive.org:survey from Sun Jun 16 04:28:55 PDT 2013 to Tue Jun 18 03:36:57 PDT 2013.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl455.us.archive.org:survey from Sun Nov 30 05:27:12 PST 2014 to Sat Nov 29 22:49:06 PST 2014.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl337.us.archive.org:survey from Thu Dec 18 00:24:41 PST 2014 to Thu Dec 18 06:57:49 PST 2014.
Topic: crawldata
3.5M
3.5M
May 16, 2018
05/18
by
Internet Archive
web
eye 3.5M
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl840.us.archive.org:survey from Tue May 15 20:39:06 PDT 2018 to Tue May 15 21:00:30 PDT 2018.
Topic: crawldata
746,035
746K
Jul 17, 2018
07/18
by
Internet Archive
web
eye 746,035
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl849.us.archive.org:survey from Tue Jul 17 16:57:11 PDT 2018 to Tue Jul 17 11:36:35 PDT 2018.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl429.us.archive.org:survey from Tue Jun 3 14:36:58 PDT 2014 to Tue Jun 3 09:53:11 PDT 2014.
Topic: crawldata
832,279
832K
Jul 17, 2018
07/18
by
Internet Archive
web
eye 832,279
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl841.us.archive.org:survey from Tue Jul 17 16:28:45 PDT 2018 to Tue Jul 17 11:24:40 PDT 2018.
Topic: crawldata
756,226
756K
Jul 17, 2018
07/18
by
Internet Archive
web
eye 756,226
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl843.us.archive.org:survey from Tue Jul 17 18:09:36 PDT 2018 to Tue Jul 17 13:08:03 PDT 2018.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl429.us.archive.org:survey from Thu Dec 25 10:57:46 PST 2014 to Thu Dec 25 05:19:18 PST 2014.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl453.us.archive.org:survey from Mon May 20 12:23:36 PDT 2013 to Tue May 21 17:29:45 PDT 2013.
Topic: crawldata
879,980
880K
Jan 11, 2019
01/19
by
Internet Archive
web
eye 879,980
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl817.us.archive.org:survey from Thu Jan 10 20:56:16 PST 2019 to Thu Jan 10 18:51:35 PST 2019.
Topic: crawldata
928,610
929K
Jan 22, 2019
01/19
by
Internet Archive
web
eye 928,610
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl344.us.archive.org:survey from Tue Jan 22 12:26:21 PST 2019 to Tue Jan 22 07:21:07 PST 2019.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl836.us.archive.org:survey from Thu Jun 15 14:25:49 PDT 2017 to Sun Jun 25 22:27:24 PDT 2017.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl453.us.archive.org:survey from Wed May 29 17:38:37 PDT 2013 to Thu May 30 11:17:05 PDT 2013.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl421.us.archive.org:survey from Sat Jan 9 03:10:06 PST 2016 to Sat Jan 9 11:11:11 PST 2016.
Topic: crawldata
894,890
895K
Jan 29, 2019
01/19
by
Internet Archive
web
eye 894,890
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl825.us.archive.org:survey from Mon Jan 28 23:46:07 PST 2019 to Mon Jan 28 17:26:58 PST 2019.
Topic: crawldata
1.1M
1.1M
Jan 15, 2019
01/19
by
Internet Archive
web
eye 1.1M
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl825.us.archive.org:survey from Tue Jan 15 00:32:09 PST 2019 to Tue Jan 15 02:46:53 PST 2019.
Topic: crawldata
1.2M
1.2M
Jan 29, 2019
01/19
by
Internet Archive
web
eye 1.2M
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl835.us.archive.org:survey from Tue Jan 29 00:07:09 PST 2019 to Mon Jan 28 17:38:10 PST 2019.
Topic: crawldata
414,228
414K
Feb 28, 2019
02/19
by
Internet Archive
web
eye 414,228
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl824.us.archive.org:survey from Thu Feb 28 04:49:14 PST 2019 to Wed Feb 27 23:44:56 PST 2019.
Topic: crawldata
2M
2.0M
Jan 22, 2019
01/19
by
Internet Archive
web
eye 2M
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl339.us.archive.org:survey from Tue Jan 22 09:37:30 PST 2019 to Tue Jan 22 03:52:56 PST 2019.
Topic: crawldata
2M
2.0M
Jan 12, 2019
01/19
by
Internet Archive
web
eye 2M
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl818.us.archive.org:survey from Sat Jan 12 11:20:21 PST 2019 to Sat Jan 12 08:14:51 PST 2019.
Topic: crawldata
710,049
710K
Jun 4, 2019
06/19
by
Internet Archive
web
eye 710,049
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl859.us.archive.org:survey from Sat May 25 15:01:10 PDT 2019 to Fri May 31 08:11:48 PDT 2019.
Topic: crawldata
617,537
618K
Jan 24, 2019
01/19
by
Internet Archive
web
eye 617,537
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl818.us.archive.org:survey from Wed Jan 23 22:13:53 PST 2019 to Wed Jan 23 15:49:31 PST 2019.
Topic: crawldata
1.9M
1.9M
Jan 10, 2019
01/19
by
Internet Archive
web
eye 1.9M
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl344.us.archive.org:survey from Thu Jan 10 16:19:12 PST 2019 to Thu Jan 10 14:12:18 PST 2019.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl809.us.archive.org:survey from Tue Dec 16 18:48:26 PST 2014 to Thu Dec 18 06:26:00 PST 2014.
Topic: crawldata
559,420
559K
Feb 2, 2019
02/19
by
Internet Archive
web
eye 559,420
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl824.us.archive.org:survey from Fri Feb 1 21:02:15 PST 2019 to Fri Feb 1 14:30:43 PST 2019.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl335.us.archive.org:survey from Wed Dec 17 17:45:22 PST 2014 to Thu Dec 18 04:19:55 PST 2014.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl426.us.archive.org:survey from Sat Aug 1 00:48:47 PDT 2015 to Sat Aug 1 05:36:22 PDT 2015.
Topic: crawldata
908,671
909K
Feb 20, 2019
02/19
by
Internet Archive
web
eye 908,671
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl339.us.archive.org:survey from Wed Feb 20 03:46:02 PST 2019 to Wed Feb 20 01:06:00 PST 2019.
Topic: crawldata
573,514
574K
Jan 27, 2019
01/19
by
Internet Archive
web
eye 573,514
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl817.us.archive.org:survey from Sun Jan 27 09:36:26 PST 2019 to Sun Jan 27 07:50:51 PST 2019.
Topic: crawldata
695,313
695K
Aug 6, 2019
08/19
by
Internet Archive
web
eye 695,313
favorite 0
comment 0
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl835.us.archive.org:survey from Thu Aug 1 03:58:02 PDT 2019 to Mon Aug 5 21:37:44 PDT 2019.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl452.us.archive.org:survey from Sat Jun 8 06:47:03 PDT 2013 to Tue Jun 11 09:38:23 PDT 2013.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl815.us.archive.org:survey from Sat Aug 1 00:44:26 PDT 2015 to Sat Aug 1 11:53:01 PDT 2015.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl837.us.archive.org:survey from Sat Aug 1 00:43:49 PDT 2015 to Sat Aug 1 11:37:56 PDT 2015.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl455.us.archive.org:survey from Wed Dec 17 15:06:59 PST 2014 to Thu Dec 18 02:12:26 PST 2014.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl836.us.archive.org:survey from Sat Aug 1 00:43:43 PDT 2015 to Sat Aug 1 11:44:39 PDT 2015.
Topic: crawldata
